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The Me t a- Analysis of Selected Interventions 



Introduction 

Statement of the Problem 

Tha provision of educational services to special 
populations typically involves the development and use of 
unique curricula, materials, and techniques and raises the 
question of the e t f ec t iveness of these procedures (Kavale & 
Glass* 1982). Nowhere are these issues more important than 
in the case of interventions with cerebral palsied children 
who cover "the entire spectrum of developmental disabilities 
mental retardation, learning disabilities, seizure 
disorders, and motor deficits" (Vining, Accardo , Rubenstein, 
Farrell, & Roizen, 1976, p. 643). 

Therapeutic interventions for children with cerebral 
palsy are aimed at improving mobility, communication, and 
educational performance. They vary widely in their emphases 
but commonly inc > ie neuromuscular facilitation and other 
physical and occupational therapy approaches, surgery, 
positioning and adaptive devices, biofeedback, and medication 
(Lord, 1984). Faced with this wide array of research 
findings , investigators conducting traditional literature 
reviews find that many questions remain unanswered. Not only 
is there a wide range of interventions and outcome var tables 
reported in the literature; but as Kavale and Glass (1982) 



have indicated with respect to special education in general, 
findings are often conflicting and variable 

It is within such a framework that meta-analysis is 
appropriately employed. Meta-analysis is a relatively recent 
development in the integration of research findings. The 
term "meta-analysis" was coined by Glass (1976) who used it 
to refer to the "analysis of analyses." (p. 3). It employs 
quantitative methods in order to provide a systematic means 
of synthesizing research findings and to answer general 
questions concerning the results of a large body of research 
literature on a specified topic. 

Purpose of the Study 

The purpose of this study was twofold. It stemmed in 
part from this researcher's interest in further developing 
research skills through the application of meta-analysis. 
Thus, the process of meta-analysis was examined, and formulae 
for computation of ffect sizes under varying conditions we^i 
applied . 

An equally important function of the study was the 
systematic examination of data regarding the interventions 
currently employed for children with cerebral palsy. Of 
particular interest were findings that might indicate which 
intervention strategies have merit and whether and under what 
conditions a given treatment is more appropriate than others. 



The study was also intended to identify interventions 
requiring fur t her empirical study. 

Research Questions 

Given the general purposes of tht study, the following 
research questions were developed • 

1« Are appropriate meta-analysis techniques available 
for use in integrating literature related to interventions 
with special populations (i.e., children with cerebral 
palsy)? In particular, can meta-analytic techniques 

accommodate control-group-only and single-subject designs? 

2. To what extent does the existing literature on 
interventions for children with cerebral palsy provide data 
needed for carrying out a meta-analysis? 

3. Does the met a- ana lysis of research on interventions 
for children with cerebral palsy indicate the relative merits 
of the various treatments? 

4. Do other factors (such as research design, sample 
size, method of calculation, year of publication) have a 
bearing on the obtained effect size? 

Methods 

The general approach to conduct ing this meta-analysis 
was the one suggested by Kavale and Glass (1981). Studies to 
be included in the meta-analysis were located and retrieved 
from a variety of sources. Literature searches were 
conducted in ERIC, Current Index to Journals in Education, 



Psychological Abstracts, Dissertation Abstracts, Index 
Medicus, and a Medline computer search. In addition, 
traditional literature reviews were consulted tc identify 
further research scudies. F in ally, several unpublished 
studies were made available for possible inclusion in the 
meta-analysis. For each of the studies finally included in 
the meta-analysis, information was retrieved and recorded on 
a data collection sheet, an example of which is found in 
Appendix A. 

The metric selected for use in the meta-analysis was 
effect size. Effect sizes were computed or estimated using a 
variety of formulae suggested in the literature. The formula 
employed depended in each case on the nature of the data 
available in a given study. 

In general, the effect size is calculated by dividing 
the difference between experimental and control group meane 
by the wi t hin-gi. uup standard deviation (Hunter, Schmidt, & 
Jackron, 1982) as given by the following formula: 

\- *c 
d - 

S 

McGaw and Glass (1980) addressed the problem of using 
final status scores to obtain mean differences. One 
difficulty arises when such scores are not available, as when 
gain scores are used. Another problem is encountered when 
there are pretreatment group differences. The authors 
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stated, "The use of gain scores, residual scores, and 
covariance adjustments when there are preexperimental group 
differences is an attempt to render the groups compa r abl e . H 
(p. 328). They have offered formulae for calculating mean 
differences with each of these three types of scores. When 
pre treatment differences do not exist, mean differences 
should be the same regardless of the scale used for 
computation; but where such differences do exist, the 
formulae will correct for them. Thus, it would seem that 
McGaw and Glass are recommending the routine use of these 
formulae father than the traditional formula which uses the 
final status score. The following are formulae given by 
McGa«? and Glass : 

For raw gain scores, 

~Gt " G c « (Y t - Y c ) - X c ) 

For residual gains, 

- - T c ) - (i - V )i\-T c ) 

For covariance adjustments, 

In these formulae, Gt and Gq represent r»w gain scores for 
the treatment and control groups respectively, Yt and Y c 
represent posttest scores of these groups, Xt and X c 
represent pretest scores, g t and g c represent residual gain 
scores, and b is the pooled wi thin-groups estimate of the 
regression of final status on initial status. 
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The final status score standard deviation was advocated 
as the measure of choice for use in the denominator in order 
to arrive at the effect size (McGaw & Glass, 1980). As Glass 
(1977) pointed out, the control group standard deviation is 
preferred when within-group standard deviations are not 
homogeneous . Furthermore , the use of standard deviations 
derived from any of the other metrics such as raw gains, 
residual gains, and covariance adjustments would all yield 
different effect sizes. The re commend at ion made by McGaw and 
Glass (1980) standardizes the mean differences on the basis 
of final status scores. 

A number of other instances arise in which it is 
necessary to use altered formulae for calculating the control 
group standard deviation or for estimating a value: when a 
direct measure is unobtainable. The situations discussed by 
McGaw and Glass (1980) include those in which (1) one or more 
treatment conditions are present but there is no control 
group, (2) final status scores have been adjusted, (3) 
factorial designs are employed with two-way or higher 
interactions, and (4) repeated measures analyses are used. 
Formulae appropriate in each of these situations were given 
by the authors and were employed as appropriate in the 
present study. 

Holmes (1984) provided several methods for computing or 
estimating effect sizes in the absence of group means and 
standard deviations. A number of alternatives exist 



depending on the information given in a study. For instance, 
an effect size can be estimated when a _t value, the degrees 
of freedom, and the size of the experimental group mean 
relative to the control group mean are known. In the case of 
an independent _t- test and under the assumption of equal size 
groups and homogeneous variances, the formula is 



2t 

ES * 

|pr 

where _t is the obtained _t value and N is the sample size. A 
correction when the test is a dependent _t-test yields the 
f allowing formula: 




fir 

where r is the correlation between the outcome variable and 
the matching variable. When the t_ value is given and group 
sizes are known but unequal and homogeneity of variance is 
assumed, the obtained _t value can be used in the formula 
below to estimate the eifect size: 




where n and n are the sample sizes of the experimental and 
control groups. When the obtained _t value is not known but the 
significance level is given, the same formula can be used to 



estimate a minimum effect size by substituting the minimum t 
value needed for significance. 

Effect sizes can also be estimated under a variety of 
other conditions, such as when (1) a critical F value is 
reported when 2 or more groups ar* compared, (2) an analysis 
of variance summary table is provided, (3) group sizes and 
the proportions of the groups possessing an attribute are 
reported, and (4) group sizes and nonparametric statistics 
(such as chi-square) are reported. Holmes (1984) has offered 
conversion or estimation formulae in each of these instances. 
Again, these formulae were employed as appropriate in the 
study. 

Glass, MoGaw, and Smith (1981) discussed the use of 
probit transformation which is an attempt to recover 
underlying metric information when only dichotomous data are 
available. Ihev provided a table on page 139 which allows 
for the quick determination of effect size when proportions 
for experimental and control conditions are available. 

An additional problem in effect size estimation is posed 
when studies under consideration include single-subject 
designs. Such a situation occurs frequently in studies of 
interventions for children with cerebral palsy. Gingerich 
^1984) has applied meta-analysis techniques in the 
aggregation of single-subject studies in order to provide 
stronger support for conclusions regarding treatments . Thus , 
single- subject designs might appropriately be included in 
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meta-analyses through the aggregation of like studies or 
several single-subject outcomes in a given study in order to 
calculate effect sizes. 

An alter native method for determining effect size in 
s ingle- sub jec t studies was employed by Prochnow-LaGrow 
(1984). She described her computational procedure in this 
way: "ES was calculated by dividing the mean change from 
baseline to intervention by intrasubject variability 
evidenced in baseline." (p. 1077). Of course, this procedure 
would be appropriate only when multiple baseline measures are 
a vai lable . 

Rosenthal (1983) offered the following formula for 
computing effect size when a correlation coefficient is 
g iven : 

2r 

d - 

Rosenthal also offered several formulae for estimating r. 

Results and Conclusions 
After reviewing over 100 studies or abstracts, 65 
studies were obtained which addressed an intervention 
strategy for children with cerebral palsy. Of these, 34 had 
to be rejected because they did not report sufficient data 
for the calculation of effect size. The remaining 31 studies 
were included in the meta-analysis. The citations for thfcse 
studies follow the references.lt should be noted at this 

erIc _ t 11 
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point that the literature retrieval was not complete due to 
temporal and monetary constraints. Tbus, the available 
research, to the extent that it Is not representative of the 
larger body of literature, may lead to bias in the 
meta-analysis results. 

Included among the 31 studies were 6 singl e- sub jec t 
designs, 14 treatment-grouo-only designs, and 11 

experimental-control group designs (6 of which involved 
random assignment). Sample sizes ranged from 1 to 124 with a 
mean sample size of 23.48. A total of 96 effect sizes was 
calculated for the 31 studies. The mean effect size was .78 
with a standard deviation of 1 .5974 . An average effect si',e 
was computed for each study (or each independent subject in a 
studv) yielding 44 effect sizes. These were averaged across 
all 31 studies, and a mean effect size of .8125 was computed 
with a standard deviation of 1.5212. 

Correlations were computed to determine if any 
relationship existed between effect size and selected study 
features. These correlations are shown in Table 1. 

Table 1 
Correlations of Effect Size 



Study Feature 


r 


Sample Size 


-.04 


Study Design 


.03 


Computational Method 


-.03 


Year of Study 


-.16 



Studies included in the meta-analysis were classified 
according to the type of intervention addressed in the 
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research. Means and standard deviations c; effect sizes were 



calculated for 


these 


c las s i f ic at ions 


. The 


results are shown 


in Table 2. 




Table 


2 






Effect Size X and SD 


by 


Intervention 


Variable # of 

Studies 


it of 
F.S 




X ES 


SD ES 


Positioning/ 
Devices 


3 


5 




.75 


0 . 4312 


Behavioral/ 
Biofeedback 


6 


26 




1.95 


1.8210 


DPM 


2 






- .23 


0 . 7 kl 2 


NDT 


3 


1 1 




. 14 


I . 5600 


Vestibular 
S t imulat ion 


2 


9 




1. 10 


1. 1734 


Other PT/OT 


8 


16 




.60 


0.8"j2 


Chronic 

Cerebellar Stim 


2 


12 




-.80 


1 . HOu 


Wrapping 


1 


k 




.99 


0. 1720 


Femoral 
Os teo t omy 


1 


3 




. 15 


0. 1543 


Medication 


1 


1 




2. 19 




Electrical Stim 


1 


1 




2.56 




Blacklight 


1 


1 




.78 





Means and standard deviations of effect sizes were also 
calculated separately for each intervention as it related to 
a given outcome variable. The results of these calculations 
are shown in Table 3. 



ERIC 



13 
- 11 - 



Table 3 



Mean and Standard Deviation of Effect Sizes 
by Intervention and Outcome Variable 



Variable 


# of 


# of 


X ES 


SD ES 




S tudies 


ES 






Head Control 










Posit ioning 


1 


2 


1.21 


0 .2750 


Biofeedback 


2 


9 


.67 


1 . 16 6 5 


Reducing Spasticity/ 










Invo lvemen t 










Positioning 


1 


2 


.58 


0.0150 


DPM 


2 


2 


.•31 


0 .8800 


NDT 


1 


1 


1 .09 




CCS 


1 


1 


- .85 




Electrical Stim 


1 


1 


2.56 




Speech/Language 










Posit ioning 


1 


1 


20 




NDT 


1 


1 


. 32 




CCS 


1 


3 


.92 


0 . 2029 


Reducing Drooling 










Biof feedback 


2 


4 


2. 28 


0 .7397 


Feeding 










Biofeedback 


1 


3 


1.78 


0 .5604 


Walking/Gait 










Biofeedback 


10 


10 


3.02 


2. 10 34 


OT/PT 


1 


3 


.05 


0 .0283 


CSS 


1 


2 


.68 


0.1750 


Motor Dev/lmprvmt 










DPM 


1 


1 


-1 .04 





NDT 


3 


8 


- .32 


1.6744 


Vestibular Stim 




8 


1.22 


1 . 1914 


PT/OT 


7 


7 


.52 


0 .6085 


CSS 


1 




-1.66 


0 .3848 


Medication 


1 




2. 19 




Mental Development 










DPM 


1 


1 ' 


- .37 




NDT 


1 


1 


2 .07 




Vestibular Stim 


1 


1 


.14 




Personal/Social 










NDT 


1 


1 


.77 




PT/OT 


1 


1 


.55 




Range of Motion 










Surgery 


1 


3 


. 15 


0 . 1543 


Wrapp ing 


1 


4 


.97 


0 . 15 30 


Breathing 










PT/OT 


1 


2 


1.59 


1 .6600 



The interpretation of effect size is straightforward and 



quite similar to interpretation of a z-score (K^vale & Glsss, 
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1981). For example, an effect size of 1.00 "would indicate 
that a subject at the 50th percentile of the control group 
would be expected to rise to the 84th percentile of the 
control group" (p. 533) following treatment. Such an effect 
size suggests that the treatment group has performed at a 
level that is one standard deviation above the control group. 

Wampler (1982) among others has cited the effect size 
values re> amended by Cohen (1977) in judging the magnitude 
of effect sizes. Using these criteria, effect sizes of .35 
and below are considered small, those between .36 and .65 are 
moderate, and those above .65 are viewed as large. Kavale 
ai?d Glass (1982), however, have indicated that effect sizes 
must be judged within the context of circumstances which may 
vary from setting to setting. 

The highest effect sizes in the present study, when 
examining intervention strategy without regard for outcome 
variable type, were found in electrical stimulation, 
medication, behavioral and biofeedback, and vestibular 
stimulation studies. When outcome variables are considered, 
neur odevel opmental therapy (NDT) and physical and 
occupational therapy fair well for selected outcomes. 
Wampler (1982) has suggested, however, that at least six 
effect sizes a r e needed in order for results to be considered 
conclusive. Thus, conclusions based on results of the present 
study are quite limited. 
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In addition to the S;aall number of effect sizes for a 
given intervention and outcome, other issues arise in 
interpretation of results. Nonindependence of effect sizes 
and variation in the quality of research, two criticisms 
lodged against meta-analysis (Glass, 1982), can be said to 
exist in the present analysis. Along with the issue of the 
biasing effect of differing sample sizes, the problem of 
nonindependence can be addressed through statistical 
procedures (Glass, 1977; Glass, McGaw, & Smith, 1981; Hedges, 
1983; Rosenthal & Rubin, 1982K 

Conclusions and Recommendations 

In response to the first research question, an 
examination of existing literature on meta-analysis revealed 
that technique? are available the accommodate a wide variety 
of research designs and data. S ingl e- sub jec t designs, common 
in special education research, can be included in 
meta-analyses if appropriate data are made available. 

The second research question was answered by calculating 
the percentage of studies, amjng those purporting to address 
an intervention strategy, which actually provided data usable 
in the meta-aaalysis . Among the 65 studies retrieved, only 
31 (48%) contained even minimal data. Thie finding 

emphasizes the need for investigators conduct ing primary 
research to provide adequate outcome data when reporting 
results. 
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The recovery of research studies was admittedly not 
exhaustive in the present meta-analysis. Present results are 
insufficient to draw conclusions regarding interventions for 



majority of interventions require further empirical study. 
Many of the retrieved studies provided inadequate information 
with regard to sample characteristics such as age and 



factors would offer greater precision in future meta-analyses 
which, in addition to including a greater number of studies, 
might more appropriately focus either on a particular 
intervention category (i.e. biofeedback or physical and 
occup* clonal therapy) or on a particular set of outcomes 
(such as motor development and improvement of motoric 
function) . 

The final research question was answered through the 
computation of appropriate correlations which revealed very 
low correlations in every instance. 



children with cerebral palsy. 



It is apparent that the 



severity of handicapping condition. 



Knowledge of these 
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